Variations of Ritz and Lehmann Bounds

Eigenvalue estimates that are optimal in some sense have selfevident appeal and leave estimators with a sense of virtue and economy. So, it is natural that ongoing searches for effective strategies for difficult tasks such as estimating matrix eigenvalues that are situated well into the interior of the spectrum revisit from time to time methods that are known to yield optimal bounds. This article reviews a variety of results related to obtaining optimal bounds to matrix eigenvalues — some results are wellknown; others are less known; and a few are new. We focus especially on Ritz and harmonic Ritz values, and rightand left-definite variants of Lehmann’s method. 1. Ritz and Related Values Let K and M be n×n real symmetric positive definite matrices and consider the eigenvalue problem Kx = λMx (1.1) Label the eigenvalues from the edges toward the center (following [16]) as λ1 ≤ λ2 ≤ λ3 ≤ · · · ≤ λ−3 ≤ λ−2 ≤ λ−1 with labeling inherited by the associated eigenvectors: x1, x2, . . . , x−2, x−1. Solutions to (1.1) are evidently eigenvalue/eigenvector pairs of the matrix M−1K, which is non-symmetric on the face of it. However, M−1K is selfadjoint with respect to the both the M-inner product, xtMx, and the Kinner product, xtKx. Denote by x the M-adjoint of a vector x, x = xtM, and by x the K-adjoint, x = xtK. “Self-adjointness” of M−1K amounts to the assertion that for all x and y, xm(M−1Ky) = (M−1Kx)my and xk(M−1Ky) = (M−1Kx)ky. Self-adjointness with respect to the Mand K-inner products implies that the matrix representation of M−1K with respect to any M-orthogonal or K-orthogonal basis will be symmetric. For a given subspace P of dimension m < n, the Rayleigh-Ritz method proceeds by selecting a basis for P, say constituting the columns of a matrix P ∈ Rn×m, and then considering the (smaller) eigenvalue problem PKPy = Λ PMPy. (1.2) This will yield m eigenvalues (called Ritz values) labeled similarly to {λi} as Λ1 ≤ Λ2 ≤ Λ3 ≤ · · · ≤ Λ−3 ≤ Λ−2 ≤ Λ−1 1 2 CHRISTOPHER BEATTIE with corresponding eigenvectors y1, y2, . . . y−2, y−1. Vectors in P given as uk = Pyk are Ritz vectors associated with the Ritz values Λk. Since R m = span(y1, y2, . . . , y−2, y−1), the full set of Ritz vectors evidently forms a basis for P, which is both K-orthogonal and M-orthogonal and may be presumed to be M-normalized without loss of generality: ukiuj = 0 and umi uj = 0 for i 6= j, and u m i ui = 1. Harmonic Ritz values [17] result from applying the Rayleigh-Ritz method to the eigenvalue problem KMKx = λKx, (1.3) which is equivalent to (1.1) — it has the same eigenvalues and eigenvectors. If we use the same subspace P, the harmonic Ritz values are then the eigenvalues of the m×m problem PKMKPy = Λ̃PKPy, (1.4) yielding Λ̃1 ≤ Λ̃2 ≤ Λ̃3 ≤ · · · ≤ Λ̃−3 ≤ Λ̃−2 ≤ Λ̃−1. Just as Ritz values are weighted means of the eigenvalues of the matrix, harmonic Ritz values are harmonic means of the eigenvalues of the matrix. Quantities which will be introduced here (for lack of a better name) as dual harmonic Ritz values result from applying the Rayleigh-Ritz method to the eigenvalue problem Mx = λMKMx, (1.5) which is also equivalent to (1.1), in the sense of having the same eigenvalues and eigenvectors. If we use the same approximating subspace P, the dual harmonic Ritz values are the eigenvalues of the m×m problem PMPy = ̃̃ ΛPMKMPy, (1.6) yielding ̃̃ Λ1 ≤ ̃̃ Λ2 ≤ ̃̃ Λ3 ≤ · · · ≤ ̃̃ Λ−3 ≤ ̃̃ Λ−2 ≤ ̃̃ Λ−1. Dual harmonic Ritz values are also harmonic means of the matrix eigenvalues, however with a different weighting than for harmonic Ritz values. Both harmonic Ritz and dual harmonic Ritz values were known even 50 years ago and found to be useful in differential eigenvalue problems — Collatz [2] referred to the harmonic Ritz problem (1.4) as Grammel’s equations (citing Grammel’s earlier work [8]) and viewed the Rayleigh quotients for the Ritz problem (1.2), the harmonic Ritz problem (1.4), and the dual harmonic Ritz problem (1.6), all as elements of an infinite monotone sequence of “Schwarz quotients” that could be generated iteratively. As long as K and M are positive definite, all three of Ritz, harmonic Ritz, and dual harmonic Ritz values provide “inner” bounds to the “outer” eigenvalues of the pencil K − λM (that is, of the problem (1.1)). In comparing the three types of approximations using the same subspace P, harmonic Ritz VARIATIONS OF RITZ AND LEHMANN BOUNDS 3 5 10 15 20 25 30 35 90 91 92 93 94 95 96 97 98 99 100 Subspace Dimension E ig en va lu e E st im at es T op o f S pe ct ru m = Ritz Values = Harmonic Ritz Values = Dual Harmonic Ritz Values 5 10 15 20 25 30 35 0 1 2 3 4 5 6 7 8 9 10 Subspace Dimension E ig en va lu e E st im at es B ot to m o f S pe ct ru m = Ritz Values = Harmonic Ritz Values = Dual Harmonic Ritz Values Figure 1. Comparison of bounds on upper and lower portions of the spectrum. values provide the best bounds of the three to the upper eigenvalues of (1.1); dual harmonic Ritz values provide the best bounds of the three to the lower eigenvalues. As an example, Figure 1 shows bounds obtained for a sequence of nested Krylov subspaces taken for P, with K = diag([1 : 2 : 100]), M = I, and a starting vector of all ones (the example of [17]). 4 CHRISTOPHER BEATTIE Theorem 1.1. Suppose K and M are positive definite. Then λk ≤ ̃̃ Λk ≤ Λk ≤ Λ̃k for k = 1, 2, . . . ̃̃ Λ−l ≤ Λ−l ≤ Λ̃−l ≤ λ−l for l = 1, 2, . . . Proof: The min-max characterization yields λk = min dimS=k max x∈S xtKx xtMx ≤ min dimS=k S⊂P max x∈S xtKx xtMx = min dimR=k max y∈R ytPtKPy ytPtMPy = Λk, and likewise, λk = min dimS=k max x∈S xtKM−1Kx xtKx ≤ min dimS=k S⊂P max x∈S xtKM−1Kx xtKx = min dimR=k max y∈R ytPtKM−1KPy ytPtKPy = Λ̃k. A similar argument shows λk ≤ ̃̃ Λk. By repeating the argument for the eigenvalue problem −Kx = (−λ)Mx, one finds −λl(−K, M) ≤ −Λ−l (where λ(A, B) is used to denote an eigenvalue of the pencil A − λB). Notice that −λl(−K, M) = λ−l(K, M). Thus, Λ−l ≤ λ−l and Λ̃−l ≤ λ−l. For any x ∈ Rn, the Cauchy-Schwarz inequality implies (xKx) =(xKMMx) ≤ xKMKx xMx and (xMx) =(xMKKx) ≤ xMKMx xKx Thus, xtMx xtMK−1Mx ≤ xtKx xtMx ≤ xtKM−1Kx xtKx , which then implies for each k = 1, 2, . . . , m 0 < λk ≤ ̃̃ Λk = min dimS=k S⊂P max x∈S xtMx xtMK−1Mx ≤ min dimS=k S⊂P max x∈S xtKx xtMx = Λk ≤ min dimS=k S⊂P max x∈S xtKM−1Kx xtKx = Λ̃k The situation is somewhat different if K is indefinite. The Ritz estimates are still “inner” bounds, that is λk ≤ Λk and Λ−l ≤ λ−l. However, both harmonic Ritz and dual harmonic Ritz values now provide “outer” bounds (lower bounds) to negative eigenvalues of (1.1) and no simple relationship is known that would predict which of the three bounds is best (essentially owing VARIATIONS OF RITZ AND LEHMANN BOUNDS 5 to there being no simple analog of the Cauchy-Schwarz inequality for indefinite inner products). Despite the differences in behavior described above, Ritz, harmonic Ritz, and dual harmonic Ritz values each provide optimal bounds – obviously each with respect to a slightly different notion of optimality. For the Ritz problem, the matrices PtKP and PtMP provide a “sampling” of the full matrices K and M on the subspace P. Whatever spectral information about the original eigenvalue problem (1.1) that we are able to deduce by examining the RayleighRitz problem (1.2) we must draw the same conclusions for all matrix pencils that are “aliased” by the Rayleigh-Ritz sampling. Define the following set of such n× n matrix pairs: